it can be shown that \(f\) is linear if and only if there exists a matrix \(A\) such that \(f(x) = Ax\) for all \(x\)
Linear Transformation
Some examples
We consider how a given matrix transforms in \(\mathbb R ^2\)
Let \(\begin{bmatrix} 2 & 1 \\ -1 & 1 \end{bmatrix}\)
it transforms the vector \(x = \begin{bmatrix} 1 \\ 3 \end{bmatrix}\) to the vector \(y = \begin{bmatrix} 5 \\ 2 \end{bmatrix}\) through the matrix multiplication
We can observe that applying the transformation \(AB\) on the vector \(x\) is the same as first applying \(B\) on \(x\) and then applying \(A\) on the vector \(Bx\).
the matrix product \(AB\) is the composition of the matrix transformations \(A\) and \(B\)
first apply transformation \(B\) and then transformation \(A\)
notice that \(AB\) is generally not equal to \(BA\)
Matrix multiplication as composition
Linear compositions - examples
Let \(A\) be the \(90^{\circ}\) clockwise rotation matrix given by \(\begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix}\)
Let \(B\) be a shear matrix along the x-axis given by \(\begin{bmatrix} 1 & 2 \\ 0 & 1 \end{bmatrix}\)
visualize how a grid of points changes when we apply the transformation \(AB\) and then compare it with the transformation \(BA\)
Matrix multiplication as composition
Linear compositions - examples
Shear then rotate
Matrix multiplication as composition
Linear compositions - examples
Rotate then shear
it demonstrates that the transformation \(AB\) is not the same as the transformation \(BA\)
Matrix multiplication as composition
Linear compositions - Iterating on a fixed map
In economics, we are often interested in analyzing behavior where we repeatedly apply a fixed matrix
For example, given a vector \(v\) and a matrix \(A\), we are interested in studying the sequence
\[
v, \quad
Av, \quad
AAv = A^2v, \quad \ldots
\]
Matrix multiplication as composition
Linear compositions - Iterating on a fixed map
Example of a sequence of iterates \((A^k v)_{k \geq 0}\) under map \(A=\begin{bmatrix} \sqrt{3}+1 & -2 \\ 1 & \sqrt{3}-1 \end{bmatrix}\)
let \(v = (-3, -3)\)
Matrix multiplication as composition
Linear compositions - Iterating on a fixed map
Example of a sequence of iterates \((A^k v)_{k \geq 0}\) under map \(A=\begin{bmatrix} \sqrt{3}+1 & -2 \\ 1 & \sqrt{3}-1 \end{bmatrix}\)
let \(v = (2.5, 0)\)
In this case, repeatedly multiplying a vector by \(A\) simply “rotates it around an ellipse”
Matrix multiplication as composition
Linear compositions - Iterating on a fixed map
Example of a sequence of iterates \((A^k v)_{k \geq 0}\) under map \(A=\begin{bmatrix} \sqrt{3}+1 & -2 \\ 1 & \sqrt{3}-1 \end{bmatrix}\)
let \(v = (-1, -0.25)\)
In this case, repeatedly multiplying a vector by \(A\) makes the vector “spiral out”
System of Linear Equations
Two equations in two unknowns
\[
\begin{aligned}
y_1 = a x_1 + b x_2 \\
y_2 = c x_1 + d x_2
\end{aligned}
\]
\[
det \begin{bmatrix}
a & b & c \\
d & e & f \\
g & h & i
\end{bmatrix}
= a(ei - fh) - b(di - fg) + c(dh - eg)
\]
If the determinant of \(A\) is not zero, then we say that \(A\) is nonsingular
Nonsingular matrices
a square matrix \(A\) has a nonzero determinant, if and only if it possesses an inverse matrix\(A^{-1}\), with the property that \(A A^{-1} =A^{-1} A = I\).
As a consequence, if we pre-multiply both sides of \(Ax = b\) by \(A^{-1}\), we get
\[
x = A^{-1} b
\]
This is the solution to \(Ax = b\) — the solution we are looking for
A more detailed explanation of matrix inverse can be found here
When the matrices have more rows than columns
\(A_{n \times k}\) matrix with \(n > k\)
this is important in many settings, including linear regression
Given arbitrary \(y \in \mathbb R ^n\), we seek an \(x \in \mathbb R ^k\) such that \(y = Ax\)
the existence of a solution is highly unlikely
focusing on the case where the columns of \(A\) are linearly independent.
the span of the columns of \(A\) is a \(k\)-dimensional subspace of \(\mathbb R ^n\).
in the \(n > k\) case we usually give up on existence
instead, seek the best approximation, looking for \(x\) that makes the distance \(\| y - Ax\|\) as small as possible
one can use either calculus or the theory of orthogonal projections
When the matrices have more rows than columns
\(A_{n \times k}\) matrix with \(n < k\)
there are either no solutions or infinitely many — in other words, uniqueness never holds
For example, consider the case where \(k=3\) and \(n=2\)
the columns of \(A\) consists of 3 vectors in \(\mathbb R ^2\)
this set can never be linearly independent
hence, one column is a linear combination of the other two
let’s say that \(a_1 = \alpha a_2 + \beta a_3\)
then if \(y = Ax = x_1 a_1 + x_2 a_2 + x_3 a_3\), we can also write
